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Executive  Summary 


With  support  from  the  ARPA/RL  Planning  Initiative  (ARPI)  we  developed  simulation, 
visualization  and  evaluation  technology  for  Air  Campaign  Planning.  We  produced  the  Air 
Campaign  Simulator  (ACS)  and  demonstrated  it  in  scenarios  involving  Korea,  Tunisia,  and  the 
Persian  Gulf  (Holmes,  1995).  A  valuable  innovation  of  ACS  was  a  representation  of  centers  of 
gravity  to  support  target  systems  analysis. 

In  1995  we  developed  the  theory  that  reasoning  involves  concepts  that  are  grounded  in  physical 
action  or  are  metaphorical  extensions  of  physical  concepts  (Cohen,  1997;  Cohen  et  ah,  1997; 
Johnson,  1987).  Supported  by  HPKB,  we  showed  how  concepts  in  Cyc’s  base-level  ontology 
are  composed  from  primitives  that  represent  physical  actions  (Cohen,  1998,  Cohen  and  Oates, 
1998).  We  developed  representations  based  in  dynamics,  instead  of  the  state-based 
representations  common  in  AI  (Bradley  and  Easley,  1997;  Campbell  and  Bobick,  1995).  We 
started  to  think  of  land  campaigns  in  terms  of  the  dynamics  of  the  battlefield,  the  ebb  and  flow  of 
influences,  the  ability  to  move  and  apply  force  (Rosenstein  and  Cohen,  1998). 

One  consequence  of  this  change  in  perspective  is  that  we  now  have  algorithms  to  search  forward 
in  a  continuous  battlespace.  We  need  not  discretize  continuous  space  and  time  -  a  perilous 
process  -  but  instead  we  can  look  forward  in  space  and  time  for  tactically  advantageous 
situations. 

In  1997  we  built  a  simulation  of  physical  behavior  called  AES,  the  Abstract  Eorce  Simulator  (see 
Section  B5).  The  basic  units  in  AES  have  physical  attributes  such  as  mass,  inertia,  and  friction 
(Atkin  et  ah,  1998).  It  is  easy  to  add  other  layers  of  attributes  to  specialize  these  units  to  a 
particular  domain,  in  particular,  simulations  of  movements  and  engagements  between  forces. 

Capture  the  Elag  (CTE)  is  a  simple  war  game  in  which  the  basic  physical  units  have  the  attributes 
of  military  units:  the  ability  to  inflict  and  absorb  damage,  the  ability  to  move,  consumption  of 
resources,  and  so  on.  The  overall  plan  was  to  use  the  Abstract  Eorce  Simulator  (AES)  and  CTE 
developed  at  Umass  to  build  and  evaluate  military  courses  of  action  (CO As).  Einer  granularity 
(e.g.,  vehicle  movements  in  ModSAE)  is  not  appropriate  for  COA  evaluation.  The  units  we 
implemented  are  automata  capable  of  limited  reactive  intelligence.  This  means  the  commander 
controls  their  activities  at  the  macro  level  (e.g.,  telling  them  to  attack  a  site)  and  the  units  figure 
out  micro-control  issues  such  as  path  planning. 

The  two  main  thrusts  of  the  completed  work  are  visualization  and  clustering  of  battlefield 
dynamics,  and  building  realistic  models  of  operational  combat  for  Capture  the  Elag. 


1.  Visualizations  of  Battlefield  Dynamics 

Military  planners  sketch  courses  of  action  in  a  stylized  way  illustrated  in  Eigure  1.  The  icons 
represent  units  and  the  tactical  engagements  between  them.  Other  icons  represent  terrain,  such 
as  mountains.  Eines  represent  natural  boundaries,  roads,  and  areas  of  responsibility.  Phase  lines 
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(denoted  PL)  are  places  that  units  meet  or  otherwise  adjust  the  timing  of  their  operations. 
Although  this  kind  of  diagram  represents  the  commander’s  intent  -  what  he  or  she  expects  to 
happen  -  it  clearly  does  not  represent  what  actually  happens  when  units  move  and  engage. 


Figure  1.  A  diagram  of  a  course  of  action,  taken  from  Army  Field  Manual  FM  101-5. 


To  grasp  the  dynamics  of  courses  of  action  such  as  the  one  in  Figure  1,  the  commander’s  staff 
engages  in  war  games.  These  laborious,  manual  exercises  typically  explore  only  a  tiny  fraction 
of  the  space  of  possible  tactical  situations.  This  is  why  we  developed  the  Capture  the  Flag 
system.  Like  a  “chess  master  in  a  box”  that  you  can  buy  at  any  games  store,  Capture  the  Flag 
includes  a  powerful  algorithm  for  playing  war  games,  as  well  as  a  simulation  of  the  dynamics  of 
military  engagements.  Humans  can  play  against  Capture  the  Flag,  or  the  system  can  play  itself, 
or  it  can  follow  -  more  or  less  strictly  -  plans  supplied  by  humans.  In  this  way,  military  courses 
of  action  can  be  explored  in  Monte  Carlo  simulation  against  an  intelligent  adversary. 


The  problem  is  to  visualize  the  results  of  hundreds  of  Monte  Carlo  trials,  each  of  which  is  a 
lengthy  time  series  of  a  vector  of  state  variables,  so  as  to  inform  the  commander  of  the  strengths 
and  weaknesses  of  a  course  of  action.  Following  an  introduction  to  the  Capture  the  Flag  system, 
we  will  describe  a  visualization  of  the  dynamics  of  simulated  courses  of  action.  This 
visualization,  called  a  probabilistic  dynamic  map,  relates  dynamical  properties  of  a  war  game  to 
the  probability  of  winning  the  game.  Sometimes,  a  game  produces  clusters  of  outcomes  that  are 
so  qualitatively  different  that  they  should  be  visualized  separately.  We  describe  an  algorithm  for 
finding  these  clusters  automatically. 

We  developed  dynamic  maps  to  visualize  the  variable  dynamics  of  Monte  Carlo  trials  in  Capture 
the  Flag,  as  well  as  the  behavior  of  mobile  robots  and  the  semantics  of  verbs  [5].  State  variables 
are  measures  of  situations  such  as  locations  and  velocities  of  units,  force  ratios,  attrition  rates, 
and  so  on.  A  trajectory  is  a  time  series  of  the  vector  of  one  or  more  state  variables.  Each 
simulation  of  a  course  of  action  generates  a  trajectory.  A  map  is  an  overlay  of  dozens  or 
hundreds  of  trajectories.  A  map  for  a  course  of  action  is  an  overlay  of  trajectories  for  that  course 
of  action.  Said  differently,  a  map  is  a  compilation  of  many  experiences  executing  a  course  of 
action.  For  example.  Figure  2a  shows  ten  engagements  between  a  Blue  unit  that  is  trying  to 
capture  a  Red  flag  and  a  Red  unit  that  is  trying  to  defend  its  flag.  Ten  trajectories  of  a  single 
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state  variable  -  distance  between  the  attacking  Blue  unit  and  the  Flag  -  are  shown.  The  colors 
denote  who  won  the  engagement;  Red  successfully  defended  its  flag  six  times.  Games  tend  to 
cluster  into  a  handful  of  qualitatively  different  engagements.  Figure  2a  shows  two  distinct  ways 
that  Red  can  win  (corresponding  to  blocking  Blue  -  the  squiggly  lines  -  and  directly  engaging 
and  attritting  Blue,  the  straight  red  lines),  and  one  way  in  which  Blue  can  win  (corresponding  to 
a  direct  engagement  that  attrits  Red). 

1.1  Probabilistic  Maps 

If  one  colors  trajectories  by  outcomes  and  lays  a  grid  over  the  map,  as  in  Figure  2a,  then  the 
proportion  of  trajectories  of  a  given  color  in  a  given  grid  cell  is  an  estimate  of  the  probability  of 
the  corresponding  outcome  given  that  the  trajectory  enters  that  cell.  More  formally,  let  x  and  ( be 
a  state  variable  and  time,  respectively,  and  let  the  coordinate  x,t  define  a  cell  of  the  map,  of  some 
fixed  area,  centered  on  the  point  x,t.  Then  the  probability  of  an  outcome,  such  as  a  win,  is 
estimated  as 

.  I  number  of  winning  trajectories  through 
total  number  of  trajectories  through  x,( 

We  can  generate  a  three-dimensional  map  from  coordinates  x,  t,  Pr(win|jic,0,  as  shown  in  Figure 
2b.  Every  cell  on  the  surface  is  a  probability  of  a  Red  win  for  a  trajectory  that  goes  through  the 
cell  defined  by  x,t. 

The  landscape  plot  in  Figure  2b  shows  a  probabilistic  map  constructed  in  this  way  for  1000  trials 
in  which  a  Blue  unit  attacks  a  Red  flag.  The  blue  spot  on  the  surface  represents  a  state  of  a 
game;  its  vertical  altitude  represents  the  probability  that  Red  will  win  in  this  state.  The  arrow 
leaving  the  blue  spot  is  the  most  common  trajectory  leaving  that  state.  You  can  see  that  Red  is 
most  likely  to  lose. 


Probability  of  Red  success 


to  objective 


Figure  2  On  the  left  (2a)  is  a  map  of  ten  trajectories,  time  series  of  distance  to  the  objective.  On  the  right  (2b)  is  a 
map  of  1000  trajectories,  showing  the  same  state  variable  plotted  against  the  probability  of  a  successful  outcome. 

The  probabilistic  map  in  Figure  2  is  “craggy,”  not  flat  or  smoothly  rising  or  falling.  Sharp  slopes 
on  dynamic  maps  correspond  to  what  commanders  call  decisive  points.  These  are  points  or 
regions  where  the  probability  of  success  changes  relatively  quickly.  The  idea  of  nonlinearity, 
particularly  nonlinear  dynamics,  is  slowly  permeating  tactical  planning  centers  in  the  U.S. 
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military,  and  can  be  seen  as  a  natural  sueeessor  to  Clausewitz,  who  revolutionized  taeties  by 
viewing  warfare  as  a  braneh  of  physics.  The  surfaee  in  Figure  2b  appeals  to  commanders 
beeause  it  provides  them  an  abstract  view  of  the  dynamics  of  the  battlefield,  one  in  which 
dangerous  areas  for  both  sides  are  explieit.  Commanders  reeognize  that  if  they  have  maps  like 
Figure  2b  and  their  opponents  do  not,  then  they  have  an  information  advantage  that  they  might 
exploit  by  “steering”  their  opponents  into  advantageous  areas  of  the  map. 

As  noted,  the  maps  in  Figure  2  represent  the  dynamies  of  engagements  between  a  single  Red  and 
a  single  Blue  unit.  Will  the  same  approach  work  -  i.e.,  produce  legible,  informative  surfaces 
relating  state  variables  to  probabilities  of  suceess  -  for  more  eomplex  seenarios,  sueh  as  the  one 
in  Figure  2?  We  have  found  no  analytical  answer  to  this  question.  In  faet,  the  same  approaeh 
works  well  for  complex  scenarios,  as  we  describe  later,  yet  fails  for  some  simpler  ones.  For 
instanee,  when  we  changed  the  seenario  for  Figure  2  a  little,  to  have  two  Blue  units  attack  a  Red 
flag  defended  by  two  Red  units,  the  resulting  map  was  unreadable.  Figure  3  shows  a 
“thresholded”  version  of  the  map:  As  before,  the  axes  represent  time  and  the  state  variable 
(although  in  this  case  the  state  variable  is  the  relative  strength  of  the  two  units)  and  the  cell 
colors  represent  probabilities.  Red  cells  represent  probabilities  of  Red  wins  greater  than  a 
threshold  value,  and  blue  cells  eorrespond  to  high  probabilities  of  Blue  wins.  There  is  no 
obvious  continuity  or  pattern  relating  these  probabilities  to  time  and  the  state  variable,  as  there 
was  in  Figure  2. 


favors  Red  favors  Blue 

mass  differential 


Figure  3. .A  probabilistic  dynamic  map  for  a  two-on-two  scenario  in  which  two  Blue  units  try  to  capture  a  Red  flag 
defended  by  two  Red  units.  The  state  variable  (horizontal  axis)  is  the  relative  strength  (or  mass)  of  the  units.  The 
vertical  axis  represents  time.  The  map  has  been  thresholded  so  all  cells  with  a  high  probability  of  win  for  Blue  are 
colored  blue,  and  similarly  for  Red.  Tactically,  the  attacking  units  might  concentrate  on  the  flag,  or  the  defending 
units,  or  split  their  efforts  between  these  goals.  This  tactical  variability  is  one  reason  that  the  map  is  not  very 
informative. 


The  reason  is  that  the  two-on-two  situation  can  unfold  in  several  qualitatively  different  ways,  and 
when  the  trajectories  for  these  different  dynamics  are  superimposed  in  a  single  map,  the  result  is 
pretty  meaningless.  It  is  as  if  we  tried  to  visualize  the  dynamics  of  several  different  pieces  of 
music,  or  normal  and  irregular  heartbeats,  or  placid  and  volatile  markets,  in  one  map.  In  any  of 
these  cases,  we  would  first  group  trajectories  that  have  similar  dynamics,  then  build  a  separate 
map  for  each  group. 
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1.2  Clustering  by  Dynamics 

We  developed  a  method  for  unsupervised  clustering  of  univariate,  categorical  time  series  called 
Bayesian  Clustering  by  Dynamics  (BCD).  Suppose  we  represent  a  military  operation  at  time  t 
with  a  state  variable  that  takes  one  of  k  values.  Between  time  t  and  time  t+1,  the  state  of  the 
operation  changes  or  remains  the  same,  but  in  either  case  can  be  represented  by  a  transition  from 
a  state  5,  e  (l... A:}  to  the  next  state  e  {l...k}.  Clearly,  k  xk  state  transitions  are  possible, 

and  the  frequencies  with  which  transitions  i,j  occur  in  a  military  operation  of  duration  N  time 
steps  can  be  stored  in  a  transition  table  like  this  one: 


a 

b 

C 

d 

e 

a 

17 

10 

0 

16 

1 

b 

0 

14 

0 

3 

2 

c 

0 

0 

3 

8 

1 

d 

0 

0 

5 

12 

0 

e 

6 

0 

0 

0 

23 

By  convention,  the  vertical  axis  represents  the  state  of  an  operation  at  time  t,  the  horizontal  axis, 
the  state  at  t+1.  Thus,  the  operation  exhibited  10  transitions  from  state  a  to  state  b,  and  16 
transitions  from  a  to  d.  The  sum  of  the  frequencies  in  a  transition  table  is  N-1  for  an  operation 
of  duration  N. 

Suppose  we  run  a  Monte  Carlo  analysis  of  100  variations  on  a  course  of  action.  We  will  have 
100  tables,  like  the  one  above,  each  representing  a  single  “playing  out”  of  the  plan  in  Capture  the 
Flag.  To  cluster  these  variations,  to  find  the  ten  or  twenty  qualitatively  different  ways  the  plan 
plays  out,  we  need  a  way  to  judge  the  similarity  of  transition  tables.  The  Kulback-Leibler  (KL) 
distance  is  such  a  method.  We  assess  the  KL  distance  for  every  pair  of  plan  instances  and  cluster 
together  those  that  are  similar.  Clustering  is  agglomerative,  which  means  we  begin  with  each 
plan  instance  in  its  own  cluster  and  repeatedly  merge  the  most  similar  clusters;  or  failing  that  the 
second  most  similar,  and  so  on;  stopping  only  when  no  further  merging  would  improve  a  quality 
metric.  The  metric  in  this  case  is  the  marginal  likelihood  of  the  data  given  the  clustering  so  far. 
The  BCD  algorithm  essentially  does  hill-climbing  in  a  space  of  possible  clusterings  of  military 
operations,  where  the  height  of  the  hill  associated  with  any  given  clustering  is  the  marginal 
likelihood. 

BCD  has  been  tested  with  data  from  Capture  the  Flag  and  with  time  series  of  robot  data, 
financial  data,  and  Bach  fugues.  In  particular,  we  used  BCD  to  cluster  data  from  81  trials  of  the 
two-on-two  scenario  that  produced  the  unsatisfactory  map  in  Figure  3.  BCD  produced  eight 
clusters.  One  of  them  contained  94%  of  the  plan  instances  in  which  Blue  won.  Three  contained 
plan  instances  in  which  Red  always  won,  but  interestingly,  the  manner  of  Red’s  victory  was  very 
different  in  the  three  clusters;  for  example,  in  one  cluster.  Red  won  by  attrition,  in  another  it 
made  a  mad  dash  to  grab  the  flag.  Four  clusters  contained  cases  in  which  Red  won  (in  different 
ways)  or  time  expired. 

Once  the  Monte  Carlo  trials  are  clustered,  one  simply  builds  a  map,  or  shows  the  trajectories,  for 
each  cluster.  Figure  4  shows  the  trajectories  that  constitute  each  of  two  clusters,  one  in  which 
Blue  always  won,  one  in  which  both  sides  won  some  engagements.  The  dynamics  of  the  clusters 
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are  quite  different;  one  can  see  why  superimposing  trajectories  from  these  (and  the  other  six 
clusters)  produces  an  illegible  map  (Fig.  3). 


favors  Red  favors  Blue 

mass  differential 


favors  Red  favors  Blue 

mass  differential 


Figure  4.  Maps  for  two  clusters  produced  by  the  BCD  algorithm.  The  state  variable,  mass  differential  (horizontal 
axis),  behaves  differently  over  time  (vertical  axis)  in  the  clusters.  On  the  left,  the  trial  begins  with  Blue  losing  mass 
through  attrition  relative  to  Red  but  in  most  cases  these  losses  “peak”  early  and  Blue  begins  to  regain  ground  relative 
to  Red.  In  three  cases,  the  differential  later  swings  back  toward  Red,  but  Blue  wins  the  games  anyway.  On  the 
right,  in  nearly  all  cases.  Blue  loses  mass  monotonically  throughout  the  game,  and  in  most  cases  loses  the  game. 


1.3  Maps  for  Complex  Scenarios 

The  complex  scenario  in  Figure  2  involves  nine  Red  and  nine  Blue  units  with  three  different 
kinds  of  dynamics:  mobile  units,  air  cavalry,  and  artillery;  and  courses  of  action  that  involve  at 
least  two  tactical  phases.  To  build  an  informative  map  for  this  scenario,  we  need  informative 
state  variables.  Because  probabilistic  maps  devote  only  one  of  three  dimensions  to  a  state 
variable,  it  is  necessary  to  design  state  variables  that  effectively  summarize  the  entire  battle. 
One,  Mass-Near- the- Action  (MNTA),  is  a  measure  of  how  much  effective  mass  is  concentrated 
near  the  center-of-mass  (COM)  of  the  opponent.  The  effective  mass  of  a  side  is  the  sum  of  the 
effective  mass  of  each  of  its  units,  and  the  effective  mass  of  a  unit  is  its  average  strength  against 
each  of  the  remaining  enemy  units.  Strength  is  a  unit's  mass  -  roughly,  how  big  it  is  -  multiplied 
by  unit-type  effects  (for  example,  artillery  has  a  multiplier  of  1.333  against  infantry).  Thus 
effective  mass  combines  the  masses  of  the  units  and  the  types  of  the  units  and  the  types  of  the 
enemy  units.  MNTA,  in  turn,  is  implemented  as  follows:  For  one  side,  say  Red,  look  at  each 
Red  unit  that  is  artillery  or  is  close  to  Blue’s  center  of  mass,  and  measure  the  effective  mass  of 
each  unit  against  all  the  Blue  blobs  that  it  is  close  to.  (Artillery  is  assumed  to  be  always  “close” 
to  the  action  because  it  can  operate  over  long  ranges.)  Roughly,  MNTA  measures  the  degree  to 
which  units  are  engaged  with  opposing  units. 

Figure  5  shows  the  trajectories  of  MNTA  for  100  Monte  Carlo  trials  corresponding  to  the 
scenario  in  Figure  2.  Trajectories  are  colored  red  if  Red  forces  captured  the  flag  in  the  south¬ 
west  comer,  blue  if  the  Red  forces  were  disabled  by  Blue,  and  green  if  time  expired.  MNTA  is 
shown  on  the  vertical  axis,  time  on  the  horizontal  axis.  Clearly,  Blue  won  most  of  the  trials.  In 
general,  trials  begin  with  Blue  MNTA  increasing  to  a  maximum  value  at  around  time  100,  then 
dropping  off  as  Red  and  Blue  units  are  destroyed  or  disabled. 
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Figure  5.  The  trajectories  of  100  trials  of  the  scenario  in  Figure  2.  Blue  lines  denote  Blue  victory,  red  lines,  Red 
victory,  and  green  lines  indicate  that  time  expired  before  either  side  won.  The  horizontal  axis  is  time,  the  vertical 
axis  is  MNTA. 


Two  probabilistic  maps  of  these  trials  are  very  informative.  Recall  that  a  probabilistic  map  for 
an  outcome,  such  as  Blue  winning,  has  three  coordinates:  a  state  variable,  time,  and  a  probability 
of  the  outcome  given  that  a  trajectory  passes  through  the  cell  defined  by  coordinate  x,t.  An 
alternative  is  to  calculate  and  plot  the  probability  of  winning  if  a  trajectory  does  not  go  through  a 
cell: 


Pr(win  I  notxd) 


number  of  winning  trajectories  that  don' t  go  through  x,t 
number  of  winning  trajectories 


Both  kinds  of  map  are  shown  in  Figure  6,  in  which  higher  probabilities  are  represented  by 
brighter  cells  (black  cells  in  Fig.  6a  and  gray  cells  in  6b  are  those  which  contained  no 
trajectories).  The  axes  of  these  figures  are  normalized  so  that  time  runs  from  0  to  100  (horizontal 
axis)  and  so  does  MNTA  (vertical  axis).  Figure  6a  is  for  the  probabilities  Pr(win  |  x,t)  and  Figure 
6b  is  for  Pr(win  |  not  x,t)-  The  former  tells  much  the  same  story  as  Figure  6,  but  the  latter  shows 
something  surprising:  A  black  cell  at  time  =  15,  MNTA  =  47.5,  corresponding  to  a  zero 
probability  of  a  Blue  win  if  Blue’s  trajectory  does  not  transit  that  cell.  To  a  military  commander. 
Figure  6b  gives  a  remarkable  imperative:  Go  through  this  cell,  or  lose!  By  avoiding  the  lighter 
cells  in  Figure  6b,  the  commander  can  trace  out  the  safer  trajectories,  because  lighter  cells 
correspond  to  higher  probabilities  of  winning  if  one’s  trajectory  doesn’t  transit  the  cell. 
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Figure  6.  Probabilistic  dynamic  maps  for  100  trials  of  the  scenario  in  Figure  2.  On  the  left,  brightness  of  a  cell  is 
proportional  to  the  probability  of  a  Blue  win  given  that  a  trajectory  transits  that  cell.  On  the  right,  brightness  is 
proportional  to  the  probability  of  a  Blue  win  if  a  trajectory  does  not  transit  a  cell. 


1.4  Nearest-neighbor  clustering 

The  nearest-neighbor  technique  allows  one  to  take  an  n-dimensional  space  of  possible  decisions 
and  determine  where  to  take  actions  that  will  lead  to  desired  outcomes.  One  possible 
visualization  from  this  work  is  a  chart  where  decision  boundaries  are  highlighted  with 
probabilities  of  outcomes  clearly  delineated. 


Discussion 

Probabilistic  dynamic  maps  represent  probabilities  of  outcomes  given  the  dynamics  of  processes. 
We  illustrated  the  idea  by  visualizing  the  probabilities  of  wins  in  war  games  given  the  dynamics 
of  state  variables  such  as  distance  to  a  goal  and  MNTA,  but  we  suspect  the  technique  is  more 
general.  Monte  Carlo  simulation  is  a  valuable  way  to  explore  a  state  space,  but  not  if  one  must 
pore  over  the  trajectories  of  state  variables  in  every  trial.  By  superimposing  these  trajectories  in 
a  map,  and  coloring  trajectories  by  outcomes,  a  great  deal  of  simulation  data  can  be  reduced  to  a 
surface  that  represents  conditional  probabilities  of  outcomes.  As  we  have  seen,  one  can 
condition  the  probabilities  on  the  event  that  a  trajectory  passes  through  a  cell,  or,  as  in  Figure  6b, 
on  the  event  that  a  trajectory  passes  through  some  other  cell. 

Some  human  art  goes  into  probabilistic  dynamic  maps,  but  we  are  working  to  automate  the 
process.  One  trick  is  to  find  an  informative  state  variable.  By  informative  we  mean  the  state 
variable  over  time  is  a  good  predictor  of  outcomes.  Finding  informative  state  variables  is  a 
statistical  model  selection  problem.  One  could  easily  use  regression  or  Bayes’  networks,  or 
related  techniques,  to  select  predictive  state  variables  were  it  not  for  the  requirement  that  the 


8 


state  variable  over  time  be  predictive.  What  we  need,  then,  is  a  characterization  of  the  dynamics 
of  a  state  variable  which  can  be  used  as  a  predictor  of  outcomes.  One  way  to  characterize  the 
dynamics  of  a  variable  is  to  cluster  the  trajectories  of  the  variable  by  their  dynamics,  as  described 
previously,  then  use  the  cluster  membership  of  a  trajectory  as  a  predictor  of  the  outcome  of  the 
trajectory.  In  a  recent  experiment,  this  approach  selected  “distance  from  the  flag”  and  “relative 
mass  of  Red  and  Blue  units”  as  the  state  variables  whose  dynamics  best  predict  the  outcome  of 
the  two-on-two  scenario  described  earlier.  These  results  are  very  preliminary  but  suggest  that 
one  aspect  of  building  probabilistic  maps  might  be  automated. 

Another  kind  of  automation  is  provided  by  the  BCD  algorithm.  Maps  can  be  difficult  to  interpret 
when  several  qualitatively  different  dynamics  are  superimposed  (Fig.  3),  but  finding 
qualitatively  different  trajectories  by  hand  is  tedious,  error-prone  work.  The  BCD  algorithm 
saves  us  the  trouble.  It  clusters  trajectories  with  similar  dynamics,  so  when  we  plot  maps  for 
each  cluster,  the  dynamics  are  much  clearer  (Fig.  4). 

It  is  equivocal  whether  maps  would  be  more  legible,  informative,  and  acceptable  to  military 
experts  if  they  did  away  with  the  time  dimension.  Phase  portraits,  which  physicists  use  to 
visualize  dynamics,  usually  plot  a  state  variable  v  at  time  t  against  v  at  t+i,  so  time  becomes 
implicit  and  relative,  but  transitions  from  state  to  state  are  clearer.  For  some  aspects  of  war 
games  absolute  time  matters,  for  others,  relative  time  suffices.  Phase  lines  (Fig.  1)  describe  both 
kinds  of  time:  Some  dictate  where  a  unit  should  be  at,  say,  1300  hours,  others  dictate  that  a  unit 
should  not  proceed  beyond  the  line  until  it  meets  another  unit.  The  explicit  time  dimension  of 
our  current  maps  makes  it  easy  to  represent  absolute  time  constraints,  but  may  create  the  illusion 
that  an  event  must  happen  at  an  absolute  time  instead  of  a  relative  time.  On  the  other  hand, 
absolute  time  constraints  are  difficult  to  represent  in  phase  portraits. 

From  the  military  perspective,  the  greatest  weakness  of  probabilistic  dynamic  maps  is  that  they 
do  not  tell  the  commander  what  he  or  she  should  do  in  a  given  state.  Clearly,  if  a  state  is 
disadvantageous,  the  commander  should  move  to  a  better  one,  but  our  maps  do  not  specify  how 
the  commander  should  do  this.  Perhaps  the  right  course  is  to  retreat,  perhaps  it  is  to  get 
reinforcements,  or  seek  high  ground,  or  engage  the  opponent.  As  time  passes  and  the 
commander  acts,  the  state  of  the  game  changes,  so  we  can  see  di  forward  relationship  between 
action  and  states  in  every  trajectory.  What  we  lack  is  a  backward  relationship  that  tells  the 
commander  what  he  or  she  should  do  to  effect  a  transition  to  a  desired  state  from  the  current 
state.  Once  this  problem  is  solved,  probabilistic  dynamic  maps  will  be  an  effective  planning 
tool. 


9 


2.  Capture  the  Flag 

The  second  major  area  of  work  was  on  the  Capture  the  Flag  system.  Capture  the  Flag  is  a 
simulator  of  war  games  and  an  automated  planner  that  commands  units  on  one  or  both  sides  of  a 
conflict.  Figure  7  is  a  screen  from  one  scenario,  modeled  after  a  training  exercise  at  the 
Command  and  General  Staff  College  at  Fort  Leavenworth.  Red  units  are  heading  toward  a  Blue 
flag  in  the  southwest  corner  of  the  map;  meanwhile,  Blue  forces  have  broken  through  a  Red  line 
in  the  northern  part  of  the  map  and  are  about  to  give  chase. 


Figure  7.  A  screen  from  Capture  the  Flag,  73  time 
steps  into  a  simulated  course  of  action.  Icons 
represent  mechanized,  infantry,  artillery,  and  air 
cavalry  units.  Colored  areas  represent  terrain: 
Light  hrown  is  open  terrain;  dark  brown  is  hilly 
terrain;  blue  is  a  lake;  green,  forest;  and  gray,  towns. 
Lines  represent  roads  and  boundaries  of  areas  of 
responsibility. 


Time:  73.0 
8.8  7.4 

Terrein:  PLAIN  )0 

R;  0.0  B:  0.0 


I  C0A-3-4-SCENARI0  ; 


In  this  scenario,  both  Red  and  Blue  forces  are  controlled  by  the  planner,  that  is,  the  machine  is 
playing  against  itself.  This  is  the  easiest  way  to  generate  Monte  Carlo  evaluations  of  courses  of 
action:  One  creates  a  version  of  the  planner  for  each  side.  Red  and  Blue;  specifying  a  course  of 
action  for  each;  and  the  planners  attempt  to  achieve  the  specified  goals  by  implementing  the 
specified  tactics.  Because  the  game  is  a  dynamical  system,  small  differences  in  the  timing  of 
actions  and  initial  placements  of  units  produce  highly  variable  outcomes. 


War  games  in  Capture  the  Flag  are  played  in  a  virtual  world  of  “nearly  three  dimensions”:  In 
addition  to  latitude  and  longitude  we  have  elevation,  although  all  hills  are  the  same  height. 
Opposing  land  and  air  units  attempt  to  capture  each  other’s  flags.  They  can  exploit  or  be 
hindered  by  terrain  features  such  as  hills,  rivers,  and  terrain  types  such  as  forest  and  swamp. 
When  units  engage,  either  directly  or  at  a  distance,  attrition  is  modeled  by  modified  Lanchester 
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equations.  In  addition,  the  dynamies  of  engagements  are  influeneed  by  models  of  psyehologieal 
faetors  -  fear,  morale,  fatigue  and  the  like. 

Opposing  units  ean  be  eontrolled  by  humans  or  by  the  GRASP  planner',  and  a  eommon 
eonfiguration  is  for  humans  to  play  against  GRASP.  A  natural  interfaee  allows  human  players  to 
direet  their  units  on  the  battlefield,  and  the  units  themselves  are  eapable  of  intelligent  reaetive 
behavior  to  earry  out  direetives  without  eonstant  supervision  (e.g.,  one  ean  direet  a  unit  to  attaek 
another  and  it  will  figure  out  how  best  to  get  there,  whieh  formations  to  adopt,  and  so  on). 
Tempo  strongly  infiuenees  the  outeomes  of  these  games.  When  human  players  lose  the  tempo, 
GRASP  presses  its  advantage,  and  a  taetieal  disadvantage  quiekly  spreads  to  a  seenario-wide  loss 
of  initiative.  The  human  beeomes  reaetive  and  eventually  loses  the  game. 


2.1  How  Capture  the  Flag  appears  to  players 

The  setup  of  a  Capture  the  Flag  seenario  is  shown  in  Figure  8.  One  panel  of  the  display  is  a  map 
with  units  arrayed  in  a  starting  eonfiguration.  (Configurations  are  seleeted  by  the  system  but  may 
be  ehanged  before  the  game  begins,  to  model  prepositioning.)  The  other  panel  shows  the 
parameters  of  the  game.  For  instanee,  one  ean  seleet  the  number  of  flags  for  eaeh  side  (Red  and 
Blue);  the  number  of  brigades  of  eaeh  of  six  kinds  (Armored,  Meeh.,  Light,  Medium,  Airborne 
and  Air  Assault);  and  the  number  of  battalions  in  eaeh  brigade.  One  ean  also  take  shorteuts  and 
add  entire  divisions  of  various  kinds,  then  modify  their  brigade  and  battalion  strueture  as  desired; 
or,  as  shown  in  the  left-hand  panel  of  Figure  9,  one  ean  speeify  eaeh  seenario  unit  individually. 
The  other  panel  in  Figure  9  shows  a  menu  to  seleet  map  views.  The  most  eommon  kind  of  view 
presents  the  military  ieons,  terrain,  roads  and  rivers,  eontrol  measures,  and  other  “bird’s  eye 
view”  features  of  the  map,  but  one  ean  see  many  other  visualizations  of  the  seenario,  as  we  will 
diseuss,  shortly. 

Floating  in  the  right-hand  panel  of  Figure  8  is  the  Control  Engine  menu.  It  is  used  to  run  and 
stop  the  simulation  and  planner;  single-step  them;  take  a  snapshot  so  one  ean  replay  a  game  from 
that  situation;  restore  a  situation;  and  zoom  in  or  out  on  the  map.  Generally,  humans  eannot 
think  fast  enough  to  win  a  Capture  the  Flag  game  in  real  time,  so  they  usually  let  the  game  run 
for  a  while  and  then  stop  it  for  a  while. 


'  The  GRASP  (General  Reasoning  with  Abstract  Physics)  planner  adapts  Lakoff  and  Johnson 
(1987)  physical  schemas  for  planning. 


11 


I  Simulation  Controls  i 


J  Division  S»»up  \r  Scenario  Units  Planner  A/  Map  \ 


tDBIue  ORed  Flags  '  .  3 

€' Armored  O  Mechanized  O  Light 
O  Medium  O  Airborne  OAirAssault 


Manuever  Brigades 
□  Battalions 


B  Aviation  Brigade 
0  Division  Artillery 

□  Engineer  Brigade 

3  Division  Cavalry  Squadron 

□  Air  Defense  Battalion 

□  Chemical  Company 


□  Enqin...  S 

• 

Q 

1^ 

Sn 

(51 

€1 

Add  Division  | 


Start  Over 


Reconfigure  Scenario  "]  J™'""  |T] 


Figure  8.  The  map  of  a  Capture  the  Flag  seenario,  a  menu  for  speeifying  the  parameters  of 
seenarios,  and  the  “eontrol  engine”  that  runs  the  simulation  and  planner. 
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Figure  9.  Menus  for  speeifying  the  units  in  a  Capture  the  Flag  seenario  and  for  configuring  the 
visualizations  presented  in  the  map. 
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2.2  The  Major  Components  of  Capture  the  Flag 


The  Capture  the  Flag  system  comprises  a  simulator  (AFS),  a  control  architecture  (HAC),  and  a 
planner  (GRASP).  In  addition,  the  system  provides  GUIs  for  humans  to  observe  and  participate 
in  war  games,  a  variety  of  visualizations,  and  instrumentation  and  other  software  support  to 
conduct  and  analyze  experiments.  One  recent  GUI  is  a  three-dimensional  display  shown  in 
Figure  10.  When  the  user  clicks  on  a  unit,  the  display  automatically  rotates  to  show  the  avenues 
of  approach,  goals,  and  threats  to  the  unit. 


Figure  10.  A  3 -dimensional  visualization.  Clicking  on  a  unit  causes  the  display  to  rotate  to  an 
angle  that  best  shows  avenues  of  approach,  the  chosen  trajectory,  and  targets  and  threats  relevant 
to  the  unit. 


Playing  against  the  system,  A  typical  experience  with  Capture  the  Flag  begins  with  the  user 
designing  a  scenario,  then  directing  his  units  (Blue,  say)  against  those  of  the  system.  The  game 
plays  out  in  the  Abstract  Force  Simulator  (AFS),  the  dynamical  analog  of  a  chessboard.  The 
simulator  implements  the  models  that  make  Capture  the  Flag  realistic;  models  of  movement; 
terrain;  visibility,  reconnaissance  and  sighting;  attrition;  and  mental  attributes  such  as  fatigue  and 
morale. 

Monte  Carlo  analysis  of  COAs.  Alternatively,  a  human  may  supply  a  course  of  action  (COA) 
and  direct  Capture  the  Flag  to  use  it  to  direct,  say.  Blue  units  in  war  games  against  the  GRASP 
planner.  Monte  Carlo  experiments  involve  fighting  numerous  games  with  roughly  the  same 
COA. 

When  a  human  player  directs  a  unit  to,  say,  attack  another,  the  Hierarchical  Control  Architecture 
(HAC)  takes  care  of  the  details: 

•  HAC  finds  the  best  route  for  the  unit  to  follow, 

•  It  avoids  obstacles  on  the  route  and  reroutes  units  dynamically 
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•  It  manages  attacks  and  maneuvers 

•  It  transforms  units  into  different  formations  as  the  tactical  situation  and  the  terrain  require. 

•  HAC  also  implements  the  “eyes  and  ears”  of  units,  and  passes  information  among  units. 

The  movements  of  opposing  forces  (Red,  say)  are  determined  by  the  GRASP  planner,  which 
coordinates  the  efforts  of  its  units,  directs  them  against  Blue  units  and  flags,  and  defends  terrain 
features  and  flags.  Just  as  human  directives  are  managed  by  HAC  and  play  out  in  AFS,  so  with 
grasp’s  directives. 


2.3  Types  and  Attributes  of  Units 

Capture  the  Flag  provides  land  and  air  combat  units  of  the  following  types:  Tank,  Mechanized 
Infantry,  Light  Infantry,  Cavalry,  Artillery,  and  Aviation.  The  smallest  units  in  Capture  the  Flag 
are  battalions,  the  largest,  divisions.  Unlike  MODSAF,  for  example,  our  Abstract  Force 
Simulator  does  not  model  the  movements  of  individual  vehicles,  but  rather,  treats  entire  units  as 
deformable  “blobs.”  The  shapes  of  blobs  change  in  response  to  terrain  and  tactical  requirements. 

We  take  Clausewitz’s  physical  metaphors  very  seriously,  so  the  most  primitive  attributes  of  units 
are  physical  attribute:  mass,  density,  shape,  velocity,  resilience,  and  elasticity.  The  Abstract 
Force  Simulator  (AFS)  determines  how  these  attributes  change  over  time  in  such  a  way  as  to 
model  warfare.  For  example, 

•  One  effect  of  attrition  is  to  reduce  the  mass  of  a  unit;  attrition  can  also  change  the  density 

of  the  unit  (the  distribution  of  its  mass  within  its  boundary),  or  its  shape. 

•  Formations  and  to  some  extent  maneuvers  involve  changing  shape.  Units  can  adopt 
columnar  and  frontal  formations,  as  well  as  wedge  (one  up,  two  back)  and  “V”  (two-up, 
one  back)  formations.  They  can  change  formation  while  moving,  and  while  engaged, 
though  in  the  latter  case  the  process  may  be  slower. 

•  The  velocity  of  a  unit  depends  on  its  shape  (formation),  the  kind  of  terrain  it  must  traverse, 

what  kind  of  unit  it  is  (e.g.,  infantry  are  faster  than  tank  units  in  wooded  terrain),  whether 
it  is  in  combat,  and  which  tactical  maneuver  it  is  following  (e.g.,  units  move  more  slowly 
when  they  share  an  avenue  of  approach). 

•  When  units  “dig  in,”  their  resilience  increases;  resilience  is  resistance  to  attrition. 

•  The  elasticity  of  a  unit  is  the  propensity  for  another  unit  to  “bounce  off’  it.  Some  tactical 

situations  require  units  to  “absorb”  others;  for  example,  a  favorite  of  Sun  Tsu’s  involves 
absorbing  an  assault,  allowing  it  to  penetrate,  then  collapsing  in  on  the  incoming  units. 

In  addition  to  these  physical  attributes  of  units,  we  model  mental  attributes  such  as  morale, 
fatigue,  and  uncertainty.  Crucially,  AFS  implements  mutual  dependencies  between  physical  and 
mental  attributes.  Thus,  attrition  reduces  morale,  and  low  morale  reduces  the  ability  of  a  unit  to 
inflict  attrition  on  other  units.  This  particular  mutual  dependency  can  lead  to  a  downward  spiral 
in  which  a  unit  becomes  progressively  more  damaged,  demoralized  and  unable  to  fight  back. 

Visibility.  Units  have  limited  sensors,  and  limited  knowledge  of  the  battlefield.  A  unit  becomes 
visible  to  another  only  when  air  reconnaissance  spots  it,  or  when  the  units  make  visual  contact. 
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Units  may  hide  in  woods  or  behind  hills.  When  humans  play  against  the  system,  they  may  seleet 
an  omniseient  view  of  all  units  (we  eall  this  eheating)  or  they  see  only  those  opposing  units  that 
have  been  sighted. 


2.4  Tactics  and  other  behaviors 

The  behaviors  of  units  range  from  low-level  physical  adjustments  (e.g.,  changing  formation)  to 
movement  and  bringing  fire,  to  tactical  plans  such  as  passage  of  lines,  to  emergent  behaviors 
such  as  the  downward  spiral  of  demoralization  mentioned  earlier.  We  have  implemented  several 
tactical  plans: 

•  For  ground  units:  Move  to  a  location,  occupy  (defend)  a  location,  retain  a  flag,  block, 

follow  and  assume,  follow  and  support,  forward  passage  of  line,  direct  attack,  indirect 
attack  (e.g.,  by  artillery). 

•  For  aviation  units,  reconnaissance,  interdiction,  close  air  support,  and  establishment  of  air 

superiority. 

We  are  constantly  implementing  other  tactical  plans;  those  on  the  agenda  include  flank  attack, 
envelopment  and  double  envelopment;  and  hasty  and  moving  defenses. 

Many  factors  affect  behavior,  and  thus  the  outcomes  of  tactical  plans.  For  example,  suppose  unit 
A  attacks  unit  B,  what  will  happen?  It  depends  on  these  and  other  factors: 

•  The  relative  masses  of  A  and  B  (big  units  tend  to  win),  and  the  types  of  the  units; 

•  The  terrain  on  which  they  engage  and  the  types  of  the  units  (e.g.,  tanks  do  poorly  against 

infantry  in  wooded  terrain); 

•  B’s  degree  of  entrenchment; 

•  The  time  it  takes  A  to  move  to  B  (this  affects  B’s  entrenchment  and  A’s  level  of  fatigue), 

which  depends  on  terrain  and  the  type  of  unit  A; 

•  Whether  or  not  A  surprises  B  (surprise  acts  as  a  multiplier  on  attrition),  which  itself 
depends  on  whether  A  is  visible  to  B; 

•  The  morale  and  fatigue  levels  of  A  and  B 

•  The  formations  of  A  and  B 

Blobology.  Many  tactical  plans  exploit  geometry,  for  example,  flank  attacks  require  exposed 
flanks.  Originally,  all  units  in  Capture  the  Flag  were  circles,  and  they  maintained  this  shape  in 
all  conditions,  making  it  difficult  to  implement  tactical  plans.  Now,  however,  units  change  shape 
in  response  to  terrain  features  (e.g.,  squeezing  through  a  pass),  tactical  requirements  (e.g., 
moving  in  column),  and  combat  situations  (e.g.,  a  wedge  formation  may  be  deformed  by  contact 
with  the  enemy).  Some  examples  are  shown  in  Figure  1 1 .  We  worked  on  the  blobology  model  in 
CTF  with  specific  emphasis  on  the  integration  with  the  current  attrition  and  movement  models  in 
CTF.  Depending  on  the  specific  shape  of  a  blob  attrition  rates  during  an  engagement  will  vary  to 
simulate  the  effects  of  a  military  unit  a  being  in  a  particular  formation  (i.e.,  delta,  line,  column). 
This  works  in  a  similar  way  for  rates  of  movement. 
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Figure  11.  Effects  on  the  morphology  of  units.  From  left:  A  unit  compressed  by  a  lake;  a  unit 
squeezing  through  a  narrow  defile  between  a  lake  and  an  impassable  area;  a  wedge  formation 
moving  east,  deformed  by  the  lake  and  impassable  area;  a  unit  moving  in  column. 


2.5  Air  Power  in  Capture  the  Fiag 

In  earlier  contracts  with  AFRL  we  developed  a  model  of  air  power  and  an  Air  Campaign 
Simulator.  This  model  was  ported  to  Capture  the  Flag.  Because  the  air  war  happens  much  faster 
than  the  land  war,  we  do  not  run  the  both  in  the  same  simulation  environment.  To  do  so  would 
slow  things  down  enormously  because  land  units  need  to  be  updated  much  less  frequently  than 
air  units,  and  checking  each  land  unit  at  the  rate  appropriate  for  air  units  is  wasteful.  One 
exception  is  attack  helicopters,  which  move  relatively  slowly  and  are  implemented  in  Capture  the 
Flag. 

A  screen  shot  for  the  air  component  of  Capture  the  Flag  is  shown  in  Figure  12. 


!  CAPTURE-THE-FIAG-BOTH-SIDES-HUMAN  ; 


igure  12.  Air  missions  in  Capture  the  Flag.  Four  mission  types  are  supported:  Air  Superiority, 
close  air  support,  reconnaissance,  and  interdiction. 
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3.  Technical  Developments  by  Year 
3.1  Technical  Results  Year  1 


For  the  Capture  the  Flag  simulator  we  augmented  the  military  unit,  terrain  and  combat  models  to 
add  the  features  necessary  to  provide  a  realistic  baseline  capability.  Examples  of  this 
augmentation  of  the  models  include  adding  differential  units  speeds  in  the  variety  of  terrain, 
differential  combat  attrition  effects  depending  on  terrain,  varying  entrenchment  rates  depending 
on  terrain,  and  elevation  effects  on  combat  attrition  and  sighting.  Also  included  in  the  new  model 
is  an  enhanced  “fog-of-war”  system  that  includes  differential  sighting  distances,  terrain  and 
elevation  effects,  sighting  on  fire  and  last-time-sighted  based  unit  positions.  The  interface  was 
enhanced  to  show  the  positions  of  units  based  on  the  fog-of-war  model  when  this  is  appropriate. 

We  also  started  building  the  defeat  modeling  mechanisms  based  on  our  discussions  with  General 
Holcombe.  The  plan  at  this  point  was  to  develop  the  models  independently  from  Capture  the 
Flag  and  run  a  series  of  experiments  to  test  their  validity.  Integration  into  CTF  will  be  the  final 
phase  of  this  work. 


3.2  Technical  Results  Year  2 

For  the  visualization  effort  we  worked  on  smoothing  surfaces  to  help  remove  noise  in  an  attempt 
to  discover  the  actual  features  of  the  scenario.  We  are  also  working  on  extending  the  existing 
data  collection  to  more  complex  scenarios  (e.g.,  more  blobs  on  either  team;  more  complex 
planning,  timing,  etc.).  We  investigated  methods  for  building  formal  models  for  particular 
physical  engagement  strategies/schemas  (e.g.,  gap  model).  This  work  was  moving  us  in  the 
direction  of  finding  relations  between  these  physical  engagement  schemas  (e.g.,  when  you  shift 
from  a  "head-on  attack"  to  "moving  through  a  pass  between  enemy  forces"). 

During  this  phase  we  had  a  series  of  very  productive  meetings  with  our  source  matter  expert. 
General  Otstott,  where  we  presented  the  latest  enhancements  to  the  CTF  models.  This  work 
included  a  more  sophisticated  combat  terrain  effects,  an  enhanced  fog-of-war  model,  and  a  better 
handling  of  multi-unit  formations.  General  Otstott  indicated  that  while  the  fog-of-war 
enhancements  contributed  positively  to  the  realism  of  the  simulation  they  fell  short  of  providing 
the  correct  sense  of  lack  of  information  present  during  an  actual  battle.  After  a  significant 
discussion  it  was  decide  that  we  needed  to  adopt  a  probabilistic  model  of  unit  sighting  that  took 
into  account  unit  types,  distances  and  terrain.  In  addition  to  solving  some  of  the  problems 
indicated  by  General  Otstott  a  new  probabilistic  model  would  integrate  nicely  with  a  possible 
future  air  power  additions  to  the  simulation. 

We  completed  a  prototype  of  the  defeat  model  and  successfully  integrated  it  into  Capture  the 
Flag.  It  currently  takes  models  attributes  such  as  courage,  fear,  health  and  moral  and  has  as  input 
parameters  rates  of  attrition,  style  of  fighting  and  isolation.  More  information  is  available  in  the 
Heeringa  paper  below  Preliminary  experimentation  indicates  that  it  adds  reasonable  effects  to  the 
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outcomes  of  simulated  engagements.  We  engaged  in  further  study  to  determine  if  we  had 
selected  a  good  set  of  attributes,  parameters  and  effects  to  model. 

We  developed  in  CTF  a  new  visualization  of  combat  force  we  call  Force  Flow.  We  simulated  the 
movement  of  forces  into  the  future  taking  terrain,  the  intentions  of  the  agents  and  the  interactions 
between  agents  into  account.  We  produced  2D  visualizations  of  the  resulting  force  data  and 
worked  briefly  on  a  3D  map-based  visualization  capability. 

We  also  designed  a  new  artillery  model  in  CTF  that  provides  for  more  realistic  control.  Artillery 
are  targeted  on  regions  using  an  implicit  spotter.  Fire  and  damage  rates  are  variable  depending  on 
the  number  of  rounds  previously  fired  at  a  region  and  number  of  tubes  available.  Requests  for 
fires  will  be  queued  and  priorities  can  be  specified.  The  first  phase  of  a  new  artillery  model 
which  allows  artillery  units  to  place  fire  in  a  specific  geometric  area  (previously  only  other  units 
could  be  targeted).  In  addition  artillery  now  undergoes  a  ranging  effect  where  the  fire  becomes 
more  accurate  over  time. 

In  the  CTF  agent  control  framework,  which  we  refer  to  as  Hierarchical  Agent  Control  (HAC), 
we  implemented  actions  whose  purpose  was  to  take  sensor  data  from  one  or  more  sources  and 
produce  an  interpretation.  We  call  these  data  combining  actions  “Abstract  Sensors”.  Abstract 
sensors  can  be  used  by  the  machine  planner  and  the  human  planner  to  build  a  high  level, 
composite  picture  from  low-level  data.  Making  an  algorithm  like  Force  Flow  into  an  abstract 
sensor  involves  specifying  who  the  generators  of  the  lower-level  data  are,  the  format  of  the 
combined  data,  and  when  and  exactly  how  the  data  will  be  produced.  We  did  this  for  Force  Flow 
and  integrated  this  new  data  source  into  the  machine  planner. 

In  the  area  of  improved  CTF  combat  models  we  began  designing  an  enhanced  blobology  model 
in  CTF.  The  new  model  simulates  blobs  as  a  collection  of  particles  connected  by  spring-like 
forces.  The  reason  for  this  effort  was  a  realization  that  our  previous  geometrically  static 
modeling  of  blobs  precluded  us  from  simulating  a  range  of  interactions  between  opposing  groups 
of  units.  Specifically,  it  was  difficult  to  accurately  simulating  a  penetration  or  pulling  two  units 
apart  “at  the  hinge”  unless  we  could  have  some  form  of  blob  deformability. 

We  enhanced  the  set  of  plans  currently  used  by  the  machine  planner  to  use  the  new  blobology 
model.  We  began  designing  a  new  layer  of  plans  that  involve  operational  goals  like  double 
envelopment  and  turning  a  flank. 

We  also  built  an  enhanced  defeat  mechanism  in  CTF.  The  original  mechanism  used  a  simple 
isolation  and  attrition  check.  The  enhanced  defeat  mechanism  uses  factors  like  courage,  fear, 
health,  morale  and  warfare  style  to  calculate  what  we  call  and  “effective  fatigue”  value.  Effective 
fatigue  is  combined  with  state  information  and  modifier  functions  to  produce  an  overall 
probability  of  surrender. 


3.3  Technical  Results  Year  3 
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We  added  additional  functionality  to  the  artillery  model  that  allows  artillery  units  to  be  placed  in 
direct  or  indirect  support  of  other  units.  In  addition,  added  an  artillery  counter  fire  model  where 
artillery  becomes  vulnerable  to  enemy  artillery  fire  when  it  fires  from  the  same  location  for  a 
certain  length  of  time. 

We  worked  on  the  blobology  model  in  CTF  with  specific  emphasis  on  the  integration  with  the 
current  attrition  and  movement  models  in  CTF.  Depending  on  the  specific  shape  of  a  blob 
attrition  rates  during  an  engagement  will  vary  to  simulate  the  effects  of  a  military  unit  a  being  in 
a  particular  formation  (i.e.,  delta,  line,  column).  This  works  in  a  similar  for  rates  of  movement. 

We  have  also  did  some  studies  of  the  stochastic  defeat  mechanism  in  CTF.  Our  preliminary 
results  lead  us  towards  a  conclusion  that  it  is  very  difficult  with  this  model  to  determine  which 
parameters  are  important  and  that  the  probabilities  being  used  to  determine  surrender  often 
combine  in  undesirable  ways.  To  rectify  these  problems  we  have  begun  investigating  a  simpler 
model  fore  defeat  which  involves  fewer  parameters  and  relies  on  less  “rolls  of  the  dice”. 

We  have  also  used  nearest  neighbor  techniques  to  cluster  data  from  simulated  battles.  This 
technique  allows  one  to  take  an  n-dimensional  space  of  possible  decisions  and  determine  where 
to  take  actions  that  will  lead  to  desired  outcomes.  One  possible  visualization  from  this  work  is  a 
chart  where  decision  boundaries  are  highlighted  with  probabilities  of  outcomes  clearly 
delineated. 

The  first  version  of  the  blobology  model  in  CTF  was  completed.  We  tested  this  model  with  a 
static  defense  scenario  in  order  to  calibrate  the  attrition  and  movement  models.  We  initiated  a 
competition  where  different  sub-groups  in  the  lab  compete  against  one  another  to  build  plans  for 
different  types  of  scenarios.  The  first  scenario  is  the  static  defense  scenario  where  the  red  force  is 
defending  a  fixed  objective  with  a  smaller  force  than  the  blue  force.  For  the  sake  of  simplicity  we 
did  not  include  artillery  plans  in  this  initial  phase  of  the  competition. 

For  the  second  phase  of  this  competition  we  tackled  a  meeting  engagement  style  scenario  and 
will  be  including  artillery  plans.  As  part  of  the  plan  competition  started  using  Tapir  to  write  plans 
in  CTF.  Tapir  is  a  semi-declarative  agent  control  language  that  was  developed  by  us  for  the  RKF 
project.  Tapir  has  constructs  for  defining  agents,  sensors,  actions,  and  messages.  It  has 
mechanisms  for  handling  multiple  agents,  re-sources,  concurrent  actions,  debugging  and 
instrumentation.  Tapir  is  built  upon  and  extends  the  Hierarchical  Agent  Control  (HAC) 
architecture. 

The  work  on  nearest  neighbor  techniques  for  clustering  data  from  simulated  battles  was 
completed.  Nearest  neighbor  queries  on  R-trees  use  a  number  of  pruning  techniques  to  improve 
the  search.  We  examined  three  common  1 -nearest  neighbor  pruning  strategies  and  generalized 
them  to  ^-nearest  neighbors.  This  generalization  clears  up  a  number  of  prior  misconceptions. 
Specifically,  we  showed  that  the  generalization  of  one  pruning  technique,  referred  to  as  strategy 
2,  is  non-trivial  and  requires  the  introduction  of  a  new  algorithm  we  call  promise-pruning.  In 
addition,  we  showed  that,  contrary  to  other  claims,  applying  this  generalized  strategy  to  ^-nearest 
neighbor  queries  results  in  a  theoretically  better  search.  This  discovery  is  reinforced  with 
empirical  results  showing  the  success  of  promise-pruning  on  both  random  and  real-world  data. 
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Enhancements  were  made  to  the  AES  and  HAC  event  systems  that  allowed  us  to  explicitly 
model  events  -  both  exogenous  and  internal  coordination  events.  This  provides  the  action/plan 
designer  the  ability  to  build  more  sophisticated  temporal  coordination  schemes.  This  work  is 
leading  towards  a  fully  realized  planning  language  where  plans  and  actions  can  be  written  to 
handled  opportunistic  events  that  occur  during  the  execution  of  a  prescribed  action. 

Along  similar  lines  we  also  worked  with  Eieutenant  General  (US  Army  Retired)  Charles  Otstott 
to  develop  realistic  actions  and  plans  for  the  CTE  simulator.  We  adapted  scenarios  from  our 
previous  HPKB  work  -  scenarios  from  the  Kansas  area  involving  division  size  units  -  and 
augmented  these  scenarios  with  better  planning  for  the  agent  and  the  adversary. 


3.4  Technical  Results  Year  4 

We  ported  CTE  to  the  new  Macintosh  operating  system  OS  X.  We  have  also  rewritten  some  of 
the  underlying  interface  protocols  in  the  simulator  to  allow  us  to  rapidly  prototype  different  types 
of  interface  interaction  schemes  and  visualizations.  The  new  interface  context  system  lets  us 
have  graphical  user  interfaces  that  alter  their  appearance  and  function  based  on  the  general 
domain  in  which  the  simulator  is  operating.  Eor  example,  in  a  the  default  CTE  simulation  mode 
the  units,  terrain  and  actions  are  all  based  on  the  domain  of  echelon  division  level  and  the  GUI 
reflects  this  by  allowing  actions,  units  and  terrain  that  are  appropriate  to  that  domain.  However, 
the  underlying  simulator  can  also  support  a  more  detailed  simulation  of  -  say  at  the  battalion 
level  down  to  individual  squads.  In  this  case  the  GUI  can  be  tailored  to  present  and  allow 
actions,  units  and  terrain  that  are  appropriate  to  that  level  of  simulation.  We  have  not  yet 
implemented  this  more  detailed  level  of  GUI  in  the  military  domain,  but  this  general  framework 
has  been  used  to  transform  CTE  into  a  simulation  of  Robots  in  an  office  environment,  a 
simulation  of  Robots  in  a  infinite  two-dimensional  plan,  a  simulation  of  Rats  doing  search  and 
rescue  operations  and  a  cellular  biology  simulation. 


4.  Products 

Capture  the  Elag  (CTE)  is  a  system  for  dynamic  planning  in  an  adversarial  domain.  It  includes  as 
components  the  Abstract  Eorce  Simulator  (AES),  a  domain-general  simulator  of  agents  applying 
forces;  the  Hierarchical  Agent  Control  architecture,  an  agent  control  architecture;  and  a  planner. 

HAC  is  a  general  toolkit  for  designing  an  action  hierarchy.  It  supports  action  abstraction,  a  multi¬ 
level  computational  architecture,  sensor  integration,  and  planning. 

Tapir  is  a  general  purpose,  semi-declarative  agent  control  language  that  extends  and  enhances  the 
HAC.  The  language  is  built  around  constructs  that  define  agents,  sensors,  actions,  and  messages. 
It  has  mechanisms  for  handling  multiple  agents,  a  flexible  resource  model,  and  multiple  means 
for  structuring  concurrent  actions. 

The  following  systems  and  prototypes  are  available  for  dissemination: 
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AFS  and  CTF.  POC:  David  Westbrook,  westy@cs.umass.edu.  AFS  and  CTF  currently  run  in 
Common  Lisp.  There  is  a  GUI  available  written  in  Macintosh  Common  Lisp.  See  http://eksl- 
www.cs.umass.edu/research/ctf. 

HAC:  POC:  David  Westbrook,  westy@cs.umass.edu.  AFS  and  CTF  currently  run  in  Common 
Lisp.  See  http://eksl-www.cs.umass.edu/research/ctf. 

Tapir:  POC:  Gary  W.  King,  gwking@cs.umass.edu.  Tapir  currently  runs  in  Common  Lisp.  See 
http://eksl-www.cs.umass.edu/research/ctf. 
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Interaction.  University  of  Massachusetts  Computer  Science  Department  Technical  Report  00-15. 
Atkin,  Marc  S.  and  Paul  R.  Cohen.  1999.  Searching  Continuous  State  Spaces  Effectively  Using 
Critical  Points.  University  of  Massachusetts  Computer  Science  Department  Technical  Report 
99-36. 


Oates,  Tim,  Zachary  Eyler-Walker  and  Paul  R.  Cohen.  1999.  Using  Syntax  to  Learn  Semantics: 
An  Experiment  in  Language  Acquisition  with  a  Mobile  Robot.  University  of  Massachusetts 
Computer  Science  Department  Technical  Report  99-35. 

Cohen,  Paul  R.  1998.  Growing  Ontologies.  University  of  Massachusetts  Computer  Science 
Department  Technical  Report  98-20. 

Schmill,  Matthew  D.  1998.  A  Distributed  Approach  to  binding  Complex  Dependencies  in  Data. 
Computer  Science,  University  of  Massachusetts  Computer  Science  Department  Technical  Report 
98-13. 


Plater,  Justus  and  Paul  R.  Cohen.  1997.  A  Randomized  ANOVA  Procedure  for  Comparing 
Performance  Curves.,  University  of  Massachusetts/Amherst.  Dept,  of  Computer  Science, 
Technical  Report  97-41. 

Oates,  Tim,  David  Jensen  and  Paul  R.  Cohen.  1997.  Automatically  Acquiring  Rules  for  Event 
Correlation  from  Event  Logs.  University  of  Massachusetts  Computer  Science  Department 
Technical  Report  97-14. 
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6.  Invited  and  Contributed  Presentations 


Paul  Cohen  attended  AAAI-99,  IDA-99  and  IJCAI-99  where  several  EKSL  papers  were 
presented.  He  also  presented  the  following  talks: 

“The  Mind  Reading  Problem.”  Invited  Speaker  at  AAAI-99  Workshop  on  Mixed  Initiative 
Intelligence,  Orlando,  FL. 

“Data  Analysis  and  the  Development  of  Robot  Minds.”  Keynote  Speaker  at  the  Third 
Symposium  on  Intelligent  Data  Analysis,  Amsterdam,  the  Netherlands. 

“Fourteen  Comments  on  Empirical  Methods.”  Invited  Speaker  at  IJCAI-99  Workshop  on 
Empirical  AI,  Stockholm,  Sweden. 

“Agent  Systems:  The  Semantic  Challenge.”  Invited  Speaker  at  Dagstuhl  Seminar  on  Agent- 
Oriented  Software  Approaches  in  Distributed  Modeling  and  Simulation:  Challenges  and 
Methodologies,  Dagstuhl  Castle,  Germany. 

“Tots  and  ‘Bots:  The  Development  of  Cognition.”  Invited  Speaker  at  the  British  Computer 
Society  Specialist  Group  on  Applied  Artificial  Intelligence  and  Knowledge-Based  Systems, 
Birbeck  College,  Eondon. 

“8.4  Comments  on  Empirical  Methods.”  Paul  Cohen,  presented  at  AAAI  Fall  Symposium, 
Falmouth  MA,  November,  1999. 

August  1999:  “Fourteen  Comments  on  Empirical  Methods,”  Keynote  Talk  presented  by  Paul 
Cohen  at  the  IJCAI-99  Workshop  on  Empirical  AI,  Stockholm  Sweden. 
http://www-eksl.cs.umass.edu/research/aaai_falll999. 

July  1999:  “Agents  Systems:  The  Semantic  Challenge,”  presented  at  Dagstuhl  Seminar:  Agent- 
Oriented  Approaches  in  Distributed  Modeling  and  Simulation,  Frankfurt  Germany, 
http  ://eksl- WWW .  cs .  umas  s .  edu/research/Dagstuhl/. 

July  1999:  “The  Mind  Reading  Problem,”  Opening  Talk  by  Paul  Cohen  at  the  Workshop  on 
Mixed  Initiative  Planning,  AAAI-99,  Orlando  EE. 
http://eksl-www.cs.umass.edu/research/miper/initiative/index.html. 

June  1999:  “Hotwash  Themes,”  presented  by  Paul  Cohen  at  DARP A/Control  of  Agent  Based 
Systems  Workshop,  Northampton  MA. 

June  1999:  “The  Grid,”  presented  by  Paul  Cohen  at  DARP  A/Control  of  Agent  Based  Systems 
Workshop,  Northampton,  MA. 

June  1999:  “Modeling,”  presented  by  Paul  Cohen  at  DARP  A/Control  of  Agent  Based  Systems 
Workshop,  Northampton,  MA. 

May  1999:  “Deep  Understanding  of  Courses  of  Action,”  presented  by  Paul  Cohen  at  the 
DARPA  Workshop  on  Knowledge  Based  Planning  for  Coalition  Forces,  Edinburgh  Scotland, 
May  1999.  http://eksl-www.cs.umass.edu/research/nato/index.html. 

February  1999:  “Building  and  Maintaining  Interactionist  Onologies,”  presented  by  Paul  Cohen 
at  AFOSR/HPKB  contractor’s  meeting.  Fort  Collins  CO. 

January  1999:  “Simulating  Tactical  Engagements  for  COA  Criticism,”  presented  by  Paul  Cohen 
at  DARP  A/High  Performance  Knowledge  Bases  Workshop,  Austin  TX. 

January  1999:  “Moving  Ahead  with  Metrics:  A  Case  Based  Approach,”  presented  by  Paul 
Cohen  at  DARP  A/High  Performance  Knowledge  Bases  Workshop,  Austin  TX. 

May  1998:  “Capture-the-Flag,  Briefing,”  presented  by  Paul  Cohen  at  the  ARPI  PI  Workshop, 
Monterey  CA. 

http://eksl-www.cs.umass.edu/research/miper/ctf/Pages/index.html. 
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April  1998:  “Physical  Planning  and  Dynamics,”  presented  by  Paul  Cohen  at  the  Information  and 
Technology  Division  Planning  Seminar  Series,  Air  Force  Research  Laboratory,  Rome  NY. 
January  1998:  “EKSL  ARPI  Work.”  Presented  at  EKSL-hosted  site  visit  for  DARPA/ARPI 
Program  Manager,  Major  Doug  Dyer. 

http://eksl-www.cs.umass.edu/research/miper/dyer/Pages/slides/index.html 

December  1997:  “Recognition  and  Prediction  Based  on  Dynamics,”  presented  by  Paul  Cohen  at 

the  HPKB  PI  Meeting,  San  Diego  CA. 

http://eksl-www.cs.umass.edu/research/hpkb/r-p-d/slides/index.html. 

June  1997:  “Semantic  Models  for  Mixed  Initiative  Plan  Evaluation  and  Repair,”  presented  by 
Paul  Cohen  at  ARPI  PI  Meeting,  San  Diego  CA. 
http://eksl-www.cs.umass.edu/research/miper/talkindex.html. 

March  1997:  “Acquiring  Conceptual  Structure  through  Activity,”  presented  by  Paul  Cohen  at 

School  of  Computer  Science,  Carnegie  Mellon  University. 

http://eksl-www.cs.umass.edu/research/conceptual-systems/index.html 

I.  Related  Service  to  the  ARPI,  DARPA  and  the  AI  Research  Community 

Editorships 

Contributions  to  DARPA  Programs 

Paul  Cohen  headed  up  the  evaluation  plan  for  the  DARPA  High  Performance  Knowledge  Bases 
(HPKB)  program  from  1997  -  1999,  and  was  a  member  of  the  design  team  for  Challenge 
Problem  specification.  Cohen  also  consulted  to  the  DARPA  Program  Managers  in  the  Rapid 
Knowledge  Eormation  program  and  the  Control  of  Agent  Based  Systems  program.  Cohen  was 
one  of  three  U.S.  academic  representatives  attending  the  DARPA  Workshop  on  Knowledge- 
Based  Planning  for  Coalition  Eorces  held  in  Edinburgh,  Scotland  in  May,  1999.  This  workshop 
brought  together  a  community  of  international  researchers,  developers  and  government 
representatives  to  discuss  emerging  advanced  planning  technology  and  how  it  might  be  further 
developed  to  support  military  coalition  planning  and  execution.  The  EKSE  also  hosted  a  4-day 
workshop  in  August  1999  for  the  EOE-1  and  EOE-2  groups  in  the  DARPA  Command  Post  of 
the  Euture  Program. 

Paul  Cohen  served  on  the  following  program  committees: 

•  Eifth  and  Sixth  International  Workshops  on  Artificial  Intelligence  and  Statistics,  1995 
and  1997; 

•  Eourteenth  National  Conference  on  Artificial  Intelligence  (AAAI-97); 

•  Eirst  and  Third  International  Symposium  on  Intelligent  Data  Analysis  (IDA-95  and  IDA- 
99). 

•  He  also  served  as  Co-Chair  of  the  Program  Committee  and  Chair  of  the  Steering 
Committee  for  IDA-97. 

Paul  Cohen  was  nominated,  and  interviewed  for  the  position  of  Director  of  DARPA’ s 
Information  Technology  Office  in  1999.  He  was  promoted  to  Pull  Professor  of  Computer 
Science  in  September,  1995,  and  was  awarded  a  University  of  Massachusetts  Paculty  Pellowship 
in  1998  for  outstanding  research  and  scholarship. 
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